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SAN DIEGO CITY SCHOOLS 

Planning, Assessment, Accountability, and Development Division 
Assessment, Research, and Reporting Team 



Executive Summary 

1995-96 PROGRESS REPORT ON THE READING RECOVERY PROGRAM 

December 16, 1996 



Issue/Concem 

The district’s Reading Recovery (RR) program has expanded over the past five years and now 
includes 38 schools. In this context, the present report documents the program’s short-term and 
sustained effects on reading achievement and its central office costs. 



Background/Discussion 

RR is a Title I-funded early intervention program that uses one-on-one tutoring to improve the 
reading skills of students experiencing serious difficulties in first grade. These students are 
identified and selected on the basis of their performance on a program-developed assessment 
instrument. 

Implementation of the district’s RR program has been problematic due to the following 
circumstances: 

1. the RR program model serves the lowest 20 percent of first-grade students while the district’s 
implementation has been limited to the lowest eight students at almost every participating 
school, serving approximately 25-33 percent of the eligible students; 

2. beginning in 1994-95, access to the district’s RR program was restricted to the most needy 
students in the lowest 20 percent while in the previous three years participants were selected 
from a range within the lowest 30 percent; and 

3. although the RR program model’s goal is to have a teacher case load of eight students per 
year, the district’s implementation has had a case load varying from 5.08 to 7.93 students 
during program expansion. 

In the present study, the short-term treatment effect of the district’s RR program was evaluated 
by determining the percentage of participants who were “discontinued” (the program’s 
designation for students who completed the required 60 RR lessons and attained their first-grade 
classmates’ average reading level) and the percentage who were “not discontinued” (the 
program’s designation for students who did not complete 60 lessons and/or did not attain their 
first-grade classmates’ average reading level). 

The sustained treatment effect of the district’s RR program was evaluated by analyzing the 
participants’ English-language arts performance in second grade and above. The study 
specifically analyzed the past four years’ midyear reading marks earned by 
discontinued students and not discontinued students. This analysis also determined the number 
and percentage of participants who were graded “moderately experienced reader” by their 
classroom teacher (superintendent’s Expectation 1 — “All students mastering reading skills and 
comprehension by grade three . . .”). 
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In addition, the central office’s costs for RR over the past five years were calculated on the basis 
of data supplied by the External Funding Budget Office. These data reflected only expenditures 
that were not part of the Title I site formula allocation. 

The major findings were as follows: 

• Short-term treatment effect 

1. The percentage of discontinued participants in 1992-93 through 1994-95 was consistent 
with corresponding percentages in other RR districts. However, it declined in San Diego 
Unified School District from 92 percent in 1993-94 to 82 percent in 1994-95. 

2. From 1992-93 through 1994-95, almost 50 percent of all not discontinued students still 
were in the program at the end of the school year with fewer than the required 60 lessons. 
Approximately 18 percent of the not discontinued students had completed at least 
60 lessons but still did not attain their classmates’ average reading level on the program’s 
assessment instrument. 

• Sustained treatment effect 

1. More than 75 percent of all discontinued students in 1991-92 and 1992-93 were 
“moderately experienced readers” in Grades 2, 3, and 4. 

2. About two-thirds of the discontinued students in 1993-94 were “moderately experienced 
readers” in Grades 2 and 3. 

3. Less than half of the discontinued students in 1994-95 were “moderately experienced 
readers” in Grade 2. 

• Central office costs 

1. Over the past five years, the costs have amounted to $4,916 per RR student (based upon 
teachers’ half-time salaries and benefits; RR teachers spend only half of their time 
tutoring participants while the other half is spent working with literacy groups of about 
15 students). 

2. The central office’s costs in 1994-95 were $6,905 per RR student (based upon teachers’ 
half-time salaries and benefits) — over $1,000 higher than the cost in any of the previous 
three years. In 1995-96, the cost decreased to $3,874 per RR student (based upon 
teachers’ half-time salaries and benefits) even though there were additional costs that 
year for training 1 1 bilingual RR teachers in the Spanish version of the program. The 
changes in cost occurred during rapid expansion of the program. In 1994-95, it expanded 
from 79 participants to 193 and from 14 teachers to 38 (including two teachers whose 
positions were funded by their sites). In 1995-96, it expanded from 193 participants to 
440 and from 38 teachers to 55.5. 

The RR program’s reduced short-term and sustained effects on students who participated in 
1994-95 might have been attributable to the following known factors: program expansion 
(24 new teachers); more stringent student selection criterion; disruptions due to the labor contract 
negotiations and strike; and interruptions due to the single track year-round calendar at some 
RR schools. Alternatively, the decline in the percentage of discontinued participants in 1994-95 
might have been an anomaly whereby the percentage in 1994-95 was more realistic than that in 
1992-93 and 1993-94 (the latter might have been overly optimistic because students over a range 
within the bottom 30 percent had been selected for participation). 
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The high cost per RR student in 1994-95 most likely was attributable to program expansion. The 
number of participating teachers increased that year to a greater degree than the increase in the 
number of participating students; consequently, each teacher was tutoring an average of 
5.08 students (the lowest value since the program’s inception in the district). This situation 
mathematically resulted in an inflated cost per RR student. 

1995-96 data relating to the RR program’s short-term and sustained effects are not included in 
this report because the necessary information is supplied by program staff up to one year late 
(after staff has obtained a compilation of the participants’ identification numbers). The district’s 
student records do not identify RR participants. Consequently, the 1995-96 program 
participation data will not become available until spring 1996-97. 



Instructional Proeram Implications 

The present study has analyzed the status of students who participated in the district’s 
RR program between 1991-92 and 1994-95 and their midyear marks in English-language arts 
content areas in Grade 2 and above. In general, the findings suggest that: 

1. the percentage of discontinued students has varied from year to year, perhaps as a result of 
the program’s problematic implementation (see page vii) and rapid expansion in the district; 

2. discontinued students (except for those in 1994-95) have shown sustained academic benefit 
from RR (i.e., at least 65 percent of these students were moderately experienced readers or 
better in Grade 2 and above); 

3. discontinued students have derived more instructional benefit from the program than 
not discontinued participants (higher percentages of the former were moderately 
experienced readers in Grade 2 and above than the latter); and 

4. the program’s cost-benefit has been impacted negatively by the sizable number of 
participants who do not complete the required 60 RR lessons and consequently are 
not discontinued. 

Although this study did not include a control group, all participants served as their own controls 
in that their “baseline” reading performance (as measured by the program’s 
Observational Survey instrument) in first grade put them in the lowest 20-30 percent of their 
classmates. It is not known how many would have achieved “moderately experienced” or above 
if they had not participated; published research findings suggest that up to 30 percent might have 
improved without intervention. 



Facilities Implications 

This report does not have any definitive facilities implications for the district. However, it is 
possible that continued expansion of the RR program during class-size reduction might result in 
additional demands for space. 



Budget Implications 

A sizable number of participants have not derived expected benefit from the RR program 
because they have not completed the required 60 lessons. Such participants comprise almost half 



of the not discontinued students (20.4 percent of all participants). Thus, the district is investing 
heavily in students who do not participate in the program long enough to fully gain the potential 
benefits. 

The present report provided evidence that the cost per student increased dramatically when the 
teacher case load declined. This evidence might have implications for plans to further expand 
the program. 



Public Support and Engaeement Implications 

Site staffs, governance teams, and school improvement councils should be informed about the 
findings of this report. 

Some of the present findings (e.g., academic data for 1994-95 participants and financial data) 
might have an impact on public perceptions and reactions toward the program. 



Recommendations 

The following actions are recommended for district staff: 

1. RR schools should identify and provide maintenance (including RR literacy groups) for 
participants who are not earning at least “moderately experienced” in reading beyond 
first grade. 

2. Investigate and implement strategies to increase the percentage of participating students who 
remain in the program long enough to be discontinued. 

3. Ensure that case loads approximate eight students per year. 

4. Eliminate the delay in receiving preliminary data regarding the percentage of 
discontinued students. 

5. Continue to consider other literacy programs for low-achieving students that might impact a 
larger number of students, be more cost effective, and serve students beyond first grade. 

6. Explore the possibility of attaining full implementation at a limited number of sites (as a pilot 
for the district) in order to assess the program’s short-term and sustained effects under such 
conditions so that future decisions about improving and/or expanding the program can be 
facilitated. 

7. Appropriately budget for continued monitoring of the RR program’s costs and benefits. 



Report prepared by Ruben Carriedo/Barry Fass-Holmes and Frank Ciriza 
RC:bf-h 
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SAN DIEGO CITY SCHOOLS 

Planning, Assessment, Accountability, and Development Division 
Assessment, Research, and Reporting Team 

1995-96 PROGRESS REPORT ON THE READING RECOVERY PROGRAM 

December 16, 1996 



Issue/Concem 

The district’s Reading Recovery (RR) program has expanded over the past five years and 
currently includes 38 schools. In this context, the present report documents the program’s 
efficacy, sustained effects on reading achievement, and central office costs. 



Background/Discussion 

RR is a Title I-funded early intervention program that employs one-on-one tutoring to improve 
reading skills of students whose performance ranks in the lowest 20 percent of their first-grade 
classmates. According to the literature, this program has been highly regarded by many 
educational researchers and is used in numerous districts across the country. However, due to 
the inherently high costs of one-on-one tutoring, administrators increasingly have questioned 
whether the relatively small number of participating students show sufficiently long-lasting 
benefit to justify the program. 

The RR program was developed in New Zealand by Marie Clay (1985) to improve the skills of 
first-grade students who experienced difficulties in reading and writing. This program identified 
prospective participants on the basis of their performance on the Observational Survey, a battery 
of RR-developed tests and observational procedures (Clay, 1994). RR’s goal was to help such 
students attain greater independence in reading and acquire the necessary skills to perform at or 
above their classmates’ average level. 

RR initially was implemented in San Diego City Schools in 1991-92 when eight teachers-in- 
training served 50 participants at four sites. In 1994-95 (the most recent year for which complete 
data were available), 193 students participated at more than 24 schools. 

Implementation of the district’s RR program has been problematic due to a host of mitigating 
circumstances, including the following: 

1. the RR program model serves the lowest 20 percent of first-grade students while the district’s 
implementation has been limited to the lowest eight students at almost every participating 
school, serving approximately 25-33 percent of the eligible students; 

2. beginning in 1994-95, access to the district’s RR program was restricted to the most needy 
students in the lowest 20 percent while in the previous three years participants were selected 
from a range within the lowest 30 percent; and 

3. although the RR program model’s goal is to have a teacher case load of eight students per 
year, the district’s implementation has had a case load that varied from 5.08 to 7.93 students 
during program expansion. 
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Interested readers can find additional background information, a review of the literature' 
(including comparisons between RR and other early intervention programs such as Success for 
All), and details in Reading Recovery — An Informal Inquiry (Evaluation Unit Report No. 701), 
Report on the Reading Recovery Program in San Diego Unified School District (Evaluation Unit 
Report No. 705), county and state reports (e.g., Holmes, 1995), and research publications (e.g., 
Hiebert, 1994). 

Goal/Purpose 

The present study was designed to address the following evaluation questions: 

• Do students who are “discontinued” (the program’s designation for successful 
completion) maintain their acquired literacy skills in the second grade and above? 

• How do students who are not discontinued perform in the second grade and above? 

• What studies have been published during the past year on RR’s sustained effects, and 
what did they find? 

• How much does the RR program cost? 

RR students’ midyear academic marks in English-language arts content areas in the second grade 
and above were analyzed to determine whether participants maintained the level of reading 
performance that they attained in the program. Results of these analyses could prove useful in 
the district’s efforts to fulfill the superintendent’s Expectation 1 — “All students mastering 
reading skills and comprehension by grade three . . .”. 

Methodology 

The present evaluation of the district’s RR program analyzed data for students who participated 
in 1991-92, 1992-93, 1993-94, or 1994-95 1 . Demographic, participation, academic, and 
financial data were used in the analyses. 

The demographic analyses determined the ethnic and gender composition of the RR program. 

The participation analyses computed the numbers and percentages of discontinued students 
(those who completed 60+ lessons and then performed at their first-grade classmates’ average 
level as measured by the program’s Observational Survey instrument) and not discontinued 
students (those who completed fewer than 60 lessons or completed 60+ lessons and then 
performed below their first-grade classmates’ average level on the Observational Survey). 

The academic analyses examined participants’ performance on the following indicators: 

• retentions in first or second grade; 

• midyear marks in reading, writing, and spelling in second grade and above (oral language 
was not analyzed because it was not a reliable indicator); and 

• participation in Special Education. 



1 1995-96 data are not included in this report because they are supplied by program staff one year 
late (after staff has obtained a compilation of the participants’ identification numbers); the 
district’s student records do not identify RR participants. Consequently, the 1995-96 data will 
not become available until spring 1996-97. 
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The financial analyses computed the average cost per participant incurred by the central office 
(i.e., expenditures that were not part of the Title I site formula allocation). RR expenses funded 
by sites were not included in the present findings. 

The students’ demographic and academic data were extracted from the district’s mainframe 
computer databases. Participation data were supplied by the RR program, and financial data 
were obtained from the Finance Division’s External Funding Budget Office. The educational 
research literature was searched for 1995 or 1996 articles dealing with RR’s long-term impact. 

Evaluation Findings 

The findings of the present study are organized around the following evaluation questions: 

• Do discontinued students maintain their acquired literacy skills in subsequent 
grade levels? 

• How do not discontinued students perform in subsequent grade levels? 

• What new information has been published about RR’s sustained effects? 

• How much does the district’s RR program cost to administer? 

To address these questions, data for four groups of participating students (Table 1) have been 
analyzed: Group 1 students were in first grade in 1991-92 (RR’s initial year in the district); 
Group 2 students were in first grade in 1992-93; Group 3 students were in first grade in 
1993-94; and Group 4 students were in first grade in 1994-95. The same students within each 
group were tracked through 1995-96 in order to evaluate their performance at each grade level. 



TABLE 1 

ANNUAL GRADE LEVELS OF THE FOUR GROUPS OF 
READING RECOVERY STUDENTS 



YEAR 


GROUP 1 


GROUP 2 


GROUP 3 


GROUP 4 


1991-92 


Grade 1 


— 


— 


— 


1992-93 


Grade 2 


Grade 1 


— 


— 


1993-94 


Grade 3 


Grade 2 


Grade 1 


— 


1994-95 


Grade 4 


Grade 3 


Grade 2 


Grade 1 


1995-96 


Grade 5 


Grade 4 


Grade 3 


Grade 2 



The present findings are organized into the following categories: 

• demographic (ethnic and gender distributions of participating students); 

• students’ status in the program (numbers who were discontinued or not discontinued); 

• retentions; 

• academic achievement in reading, writing, and spelling beyond first grade; 

• recently published findings on RR’s sustained effects; and 

• financial information. 
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Demographics : The ethnic composition of the district’s RR program is shown in Table 2. From 
1991-92 through 1994-95, less than one-fifth of the participants were White students (73 of 402), 
almost half were African American students (193 of 402), less than five percent were 
Filipino students (1 1 of 402), and about one-quarter were Hispanic students (108 of 402). 

The following ethnicities also were represented in the district’s RR program (but not included in 
Table 2 because of the small numbers): Guamanian (one student in Group 3); Hawaiian 
(one student in Group 1); Other Pacific Islander (one student in Group 3); Samoan (one student 
in Group 3); Cambodian (one student in Group 2); and Hmong (one student in Group 3). 
Group 4 included 1 1 students in the Other category. 



TABLE 2 

ETHNIC DISTRIBUTION OF STUDENTS IN THE READING RECOVERY PROGRAM 



GROUP 


N* 


WHITE 


AFRICAN 

AMERICAN 


FILIPINO 


HISPANIC 




% 


3^7/n ’^r 


% 




% 


|Kn*P 


% 


1 (1991-92 participants) 


50 




28.0 


fe20 


40.0 


fe 0 ;^ 


0.0 


llfS15r% 


30.0 


2 (1992-93 participants) 


80 


1 6 • ? F 


20.0 


$&31> 


38.8 




8.8 


%ms •:? 


31.2 


3 (1993-94 participants) 


79 


$mm 


16.4 




55.7 


§Sf37:-' 


3.8 




19.0 


4 (1994-95 participants) 


193 


lf-30¥ 


15.5 


oo 

o\ 


50.8 




<0.1 


ii;«53-- 


27.5 


TOTAL 


402 


m&iim 


18.2 


• ;> 


48.0 


WmM 


2.7 


mm 


26.9 



N= total number of students 
n=number of students 

t In some rows, the sum of the ns does not equal N due to students in ethnic groups that were not included in 
the table (see above narrative). 



The RR program’s gender composition is shown in Table 3. Overall, 44 percent of the 
participants were female (177 of 402) and 56 percent were male (225 of 402). 
Female participants constituted a smaller percentage of each group (except Group 3) than males. 



TABLE 3 

GENDER DISTRIBUTION OF STUDENTS IN THE READING RECOVERY PROGRAM 



GROUP 


N 


FEMALES 


MALES 




% 




% 


1 (1991-92 participants) 


50 


Iei9-M 


38.0 




62.0 


2 (1992-93 participants) 


80 


i?30< 


37.5 


#e - 50S 


62.5 


3 (1993-94 participants) 


79 


#&40£. 


50.6 


S1&39S? 


49.4 


4 (1994-95 participants) 


193 


§§r;.88 b? 


45.6 


S.105 v. 


54.4 


TOTAL 


402 


Kl77?‘ : 


44.0 




56.0 



n=number of students 
N=total number of students 
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Students’ Status in RR : Table 4 presents an explanation of the terminology used in Table 5, and 
the latter includes the following information for each of the four groups of students: the total 
number of participants; the total number who completed 60+ lessons; the number who were 
discontinued (60+ lessons and attained their first-grade classmates’ average reading level); the 
percentage of discontinued students relative to the number who completed the program; and the 
percentage of discontinued students relative to the total number who participated. (The last 
two rows in Table 4 are distinctive — the number who completed the program is a portion of the 
number who participated.) 



TABLE 4 

EXPLANATION OF STUDENTS’ STATUS IN THE READING RECOVERY PROGRAM 



STATUS 


EXPLANATION 


participated 


a student who took at least one lesson after scoring in the 
bottom 20 percent of her/his first-grade class on the 
Observational Survey at the beginning of the school year 


completed 


a student who finished 60+ lessons 


discontinued 


a student who completed 60+ lessons and scored at or above 
her/his first-grade classmates’ average level on the 
Observational Survey at the end of the school year 


not discontinued 


a student who completed 60+ lessons and scored below 
her/his first-grade classmates’ average level on the 
Observational Survey at the end of the school year, or a 
student who completed fewer than 60 lessons for one of 
various reasons (began the program too late in the semester, 
moved to another district or school, etc.) 


percentage discontinued 
re: completed 


the quotient of the number of students who were 
discontinued divided by the number who completed the 
RR program; this value excluded students who withdrew 
from the program 


percentage discontinued 
re: participated 


the quotient of the number of students who were 
discontinued divided by the number who participated in the 
RR program; this value included students who withdrew 
from the program 
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It is important to note the distinction between the two percentages in Table 5; both are used as 
indicators of the program’s success. According to RR proponents, the values in the fifth column 
(“percentage discontinued re: completed”; this percentage is the quotient of the number who are 
discontinued divided by the number who complete 60+ lessons) are the proper indicators because 
only students who complete 60+ lessons are fully exposed to the RR treatment. According to 
RR opponents, the values in the sixth column of Table 5 (“percentage discontinued 
re: participated”; this percentage is the quotient of the number who are discontinued divided by 
the number of participants in the program) are the proper indicators because students who do not 
complete the required 60+ lessons nevertheless incur a sizable cost to the district for their 
incomplete participation and therefore they should be counted. 

As shown in Table 5, the number of participating students more than doubled in 1994-95 relative 
to the previous years (simultaneously, the program trained 24 new teachers and employed three 
teacher leaders, one of whom was in training in 1994-95). Group 4 had the largest numbers of 
students who participated, completed 60+ lessons, and were discontinued. 

Also in Table 5, each value in the fifth column was at least 18 percentage points higher than the 
corresponding one in the sixth column. Both indicators of the program’s success showed that the 
lowest percentage of discontinued students was in Group 1 (i.e., the lowest success rate 
occurred in the program’s initial year). Groups 2 and 3 tied for the highest percentage relative to 
those who completed 60+ lessons, while Group 2 had the highest percentage relative to all 
participants. Group 4 had considerably lower values in the fifth and sixth columns in Table 5 
than Group 2 (to be discussed in more detail later in this report). 



TABLE 5 

STUDENTS’ STATUS IN THE READING RECOVERY PROGRAM 



GROUP 


n 

discontinued 


n 

completed 


n 

participated 


Sire:' completed^ 


|#re:’:participated.^ 


1 (1991-92 participants) 


19 


30 


50 






2 (1992-93 participants) 


59 


64 


80 






3 (1993-94 participants) 


49 


53 


79 






4 (1994-95 participants) 


103 


124 


193 






TOTAL 


230 


272 


402 







n=number of students 



In summary. Table 5 shows that 

• approximately seven out of ten participants (272 of 402) have completed 60+ lessons; 
and 

• approximately six out of ten participants (230 of 402) have been discontinued. 
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To account for differences between the numbers of students who participated and those who 
were discontinued (Table 5), Table 6 presents the numbers of not discontinued students and 
indicates why they were not discontinued. 

The total number of not discontinued students was highest in 1994-95 (Group 4) and it was 
almost as high in 1993-94 (Group 3) as it was in 1991-92 (the first year of the RR program; 
Group 1). Approximately two-thirds (117 of 172) of these students still were in the program at 
the end of the school year, while one-third (55 of 172) had withdrawn. 

As shown in Table 6, failure to complete the required 60 lessons was the primary reason why 
students were not discontinued. Almost 75 percent (126 of 172) of the not discontinued 
students had fewer than 60 lessons; they began the program too late in the semester, left the 
district, or moved to another school before completing 60 lessons. Twenty percent (35 of 172) of 
the not discontinued students completed 60+ lessons but did not attain their first-grade 
classmates’ average reading level on the Observational Survey at the end of the school year. 

Also shown in Table 6, a steady decrease occurred from 1991-92 (Group 1) to 1993-94 
(Group 3) in the number and percentage of not discontinued students who still were in the 
program at the end of the school year and had completed 60+ lessons. However, they increased 
in 1994-95 (Group 4). An increase occurred from 1991-92 (Group 1) to 1993-94 (Group 3) in 
the percentages of not discontinued students who still were in the program at the end of the 
school year and had completed fewer than 60 lessons. The percentage stayed about the same in 
1994-95 (Group 4). 

Only five not discontinued students withdrew from RR between 1991-92 and 1994-95 because 
they had been assigned to Special Education (Table 6). This number represented approximately 
one percent of all participants. 



TABLE 6 

REASONS WHY STUDENTS WERE NOT DISCONTINUED FROM 
THE READING RECOVERY PROGRAM 



GROUP 


N 


in program 
at year-end: 
60+ lessons 


in program 
at year-end: 
<60 lessons 


withdrew with 
60+ lessons 


withdrew with 
<60 lessons 


withdrew — 
Special 
Education 


n 


% 


n 




n 


% 


n 


% 


n 


% 


1 (RR in 1991-92) 


31 


10 


32 


13 


BUS 


i 


3 


7 


23 


0 


0 


2 (RRin 1992-93) 


21 


5 


24 


10 


00 

iSOSfe: 


0 


0 


6 


28 


0 


0 


3 (RRin 1993-94) 


30 


2 


7 


15 


§-50^ 


2 


7 


9 


30 


2 


7 


4 (RR in 1994-95) 


90 


18 


20 


44 


&*49 ‘ 


3 


3 


22 


24 


3 


3 


TOTAL 


172 


35 


20 


82 




6 


4 


44 


26 


5 


3 



N=total number of not discontinued students 
n=n umber of not discontinued students 
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The participants’ status in the program is summarized graphically in Figure 1. The graphs show 
the percentages of discontinued and not discontinued students relative to the total number of 
participants in each group. The percentage of students who were discontinued was highest in 
1992-93 (Group 2) and declined each of the past two years (Groups 3 and 4). Conversely, the 
percentage of not discontinued students with fewer than 60 lessons increased each of the past 
two years (Groups 3 and 4). The percentage of not discontinued students with 60+ lessons also 
increased last year (Group 4). 




GROUP 1: 

1991-92 participants 

H Discontinued 

l~~l Not discontinued: 60+ lessons 

□ Not discontinued: <60 lessons 

GROUP 3: 

1993-94 participants 

| Discontinued 
O Not discontinued: 60+ lessons 
ITI Not discontinued: <60 lessons 
E3 Spec Ed 




GROUP 2: 

1992-93 participants 

| Discontinued 
n Not discontinued: 60+ lessons 
□ Not discontinued: <60 lessons 

GROUP 4: 

1994-95 participants 

H Discontinued 

□ Not discontinued: 60+ lessons 

□ Not discontinued: <60 lessons 
El Spec Ed 



Figure 1. Participants’ Status in the Reading Recovery Program — Percentages. For each of the four groups, the 
graphs show the degree to which participants were discontinued or not discontinued. The slice of the pies for “Not 
discontinued: 60+ lessons” represents the percentage of participants who did not attain their first-grade classmates’ 
average reading level at the end of the program. The slice of the pies for “Not discontinued: <60 lessons” 
represents the percentage of participants who completed fewer than 60 lessons because they began the program too 
late in the semester, left the district, moved to a school that did not participate in the program, or withdrew from the 
program. 
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The numbers and percentages of discontinued and not discontinued students within the three 
largest ethnic groupings are shown in Table 7. In total, approximately 55 percent of the White, 
African American, and Hispanic students were discontinued; the exact total percentage (i.e., the 
success rate) was almost identical between these three ethnicities. Within each ethnicity, 
however, the percentage of discontinued students varied between the four groups of participants. 
The highest percentage of discontinued White and African American students was in Group 2, 
and the highest percentage of discontinued Hispanic students was in Group 3. The lowest 
percentage of discontinued students for all three ethnicities was in Group 1 (i.e., during the 
program’s first year). 



TABLE 7 

STUDENTS’ STATUS IN THE READING RECOVERY PROGRAM: 
BREAKOUT BY ETHNICITY 



ETHNICITY BY GROUP 


-H- 

z 


DISCONTINUED 


NOT DISCONTINUED 


n 


% 


n 


% 














1 (1991-92 participants) 


14 


5 


35.7 


9 


64.3 


2 (1992-93 participants) 


16 


12 


75.0 


4 


25.0 


3 (1993-94 participants) 


13 


7 


53.8 


6 


46.2 


4 (1994-95 participants) 


30 


15 


50.0 


15 


50.0 


^TWCAN:AMERlCi^ (TOTAL);^^^ 


’a93: 


1^109^ 


}0i56':5-$i> 






1 (1991-92 participants) 


20 


10 


50.0 


10 


50.0 


2 (1992-93 participants) 


31 


23 


74.2 


8 


25.8 


3 (1993-94 participants) 


44 


26 


59.1 


18 


40.9 


4 (1994-95 participants) 


98 


50 


51.0 


48 


49.0 


IroPANiCpa 


;;i08 




3£55.6i|| 




;fei744:4|i?! 


1 (1991-92 participants) 


15 


4 


26.7 


ii 


73.3 


2 (1992-93 participants) 


25 


17 


68.0 


8 


32.0 


3 (1993-94 participants) 


15 


11 


73.3 


4 


26.7 


4 (1994-95 participants) 


53 


28 


52.8 


25 


47.2 




,402.; 






•dJ 





N=total number of students for tbe row 
n=number of students 

$ In some rows, the sum of the ns does not equal N because of students in ethnic groups not included in the 
table (see top of page 4). 
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The numbers and percentages of discontinued and not discontinued students within each 
gender are shown in Table 8. Although females comprised only 44 percent of the participants 
(see Table 3), the percentage of discontinued females was almost nine points higher than the 
percentage of discontinued males. Within each gender, however, the percentage of 
discontinued students varied between the four groups of participants. The highest percentage of 
discontinued female and male students was in Group 2, while the lowest percentage of 
discontinued female and male students was in Group 1 (i.e., during the program’s first year). 



TABLE 8 

STUDENTS’ STATUS IN THE READING RECOVERY PROGRAM: 
BREAKOUT BY GENDER 



GENDER BY GROUP 


N 


DISCONTINUED 


NOT DISCONTINUED 


n 


% 


n 


% 




t®. 










1 (1991-92 participants) 


19 


8 


42.1 


11 


57.9 


2 (1992-93 participants) 


30 


25 


83.3 


5 


16.7 


3 (1993-94 participants) 


40 


29 


72.5 


11 


27.5 


4 (1994-95 participants) 


88 


48 


54.5 


40 


45.5 




“225 


jg; 120#! 


®533fi: 






1 (1991-92 participants) 


31 


11 


35.5 


20 


64.5 


2 (1992-93 participants) 


50 


34 


68.0 


16 


32.0 


3 (1993-94 participants) 


39 


20 


51.3 


19 


48.7 


4 (1994-95 participants) 


105 


55 


52.4 


50 


47.6 




402'. 


#1230;©; 


stt-v';- 


stasis 





N=total number of students for the row 
n=number of students 
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Retentions : The number of participating students who were retained in Grade 1, disaggregated 
by their status in RR, is presented in Table 9. Out of the 402 students who were in the program 
since its inception, 12 (3.0 percent) were retained in the first grade (the corresponding 
districtwide value was 1.7 percent for 1993-94, the most recent year for which data were 
available). Of these 12 students, only two (0.5 percent) had been discontinued from the 
program, eight still were in the program at the end of the school year, and two withdrew after 
completing fewer than 60 lessons. 



TABLE 9 

STUDENTS IN THE READING RECOVERY PROGRAM 
WHO WERE RETAINED IN GRADE 1 



GROUP 


n 

discontinued 


n 

in program at 
year-end: 
60+ lessons 


n 

in program at 
year-end: 
<60 lessons 


n 

withdrew with 
60+ lessons 


n 

withdrew with 
<60 lessons 


1 (RRin 1991-92) 


0 


2 


3 


0 


i 


2 (RR in 1992-93) 


1 


0 


0 


0 


0 


3 (RR in 1993-94) 


0 


0 


0 


0 


0 


4 (RR in 1994-95) 


1 


2 


1 


0 


1 


TOTAL 


2 


4 


4 


0 


2 



n=number of students who were retained in first grade 



Only one student (not discontinued) out of the 209 in Groups 1-3 was retained in Grade 2. Data 
for Group 4 retentions in Grade 2 will become available in spring 1997. 

Academic Performance — Discontinued Students : To evaluate the RR program’s long-term 
impact on achievement, midyear academic marks in reading, writing, and spelling have been 
statistically analyzed. The following sections present longitudinal data for discontinued 
students (i.e., participants who attained their first-grade classmates’ average reading level) to 
reveal whether the program might have had sustained effects. 
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• Reading: The midyear marks of discontinued students in second-, third-, fourth-, and 
fifth-grade reading are presented in Table 10. This table shows the numbers and percentages of 
discontinued students who earned at least “moderately experienced” in reading. 

Some students did not have midyear marks because 1) they left the district before completing the 
grading period, or 2) they had multiple (brief) enrollments and were not at a school sufficiently 
long to be issued marks by their teachers. Only the discontinued students who had midyear 
marks were included in the following analyses. 

As shown in Table 10, more than 70 percent of the discontinued students in Groups 1, 2, and 3, 
but not 4, earned at least “moderately experienced” in second-grade reading. The percentage of 
such students earning at least “moderately experienced” in second-grade reading decreased 
steadily from Group 1 (1991-92 participants) to Group 3 (1994-95 participants) and then 
decreased dramatically in Group 4 (1995-96 participants). These decreases will be discussed in 
detail later in the report. 



TABLE 10 

DISCONTINUED STUDENTS’ MIDYEAR READING ACHIEVEMENT 
IN SECOND GRADE AND ABOVE 



GROUP 


GRADE 


N 


ACHIEVED AT LEAST 
“MODERATELY EXPERIENCED” 


n 




1 (RR in 1991-92; 

50 participants; 19 discontinued) 


2 


16 


13 




3 


16 


13 




4 


14 


11 




5i 


14 


13 


mM:* 92$ 


2 (RR in 1992-93; 

80 participants; 59 discontinued) 


2 


42 


33 


: 78.6 ^ 


3 


36 


33 




4t 


32 


27 


84.4 


3 (RR in 1993-94- 
79 participants; 49 discontinued) 


2 


42 


30 


s* C7JL4 . 


3 f 


34 


22 




4 (RR in 1994-95; 

193 participants; 103 discontinued) 


2% 


88}* 


37 




3 


n/a 


n/a 





N=total number of discontinued students in each group for whom academic marks were available 
(the different values of N across grade levels within groups were due to students leaving the 
district temporarily or permanently). Values include students attending a site (i.e., Damall) with a 
grade-reporting waiver and its own five-category rubric for marks in reading. 
n=number of discontinued students 

* These percentages are relative to the value of N (rather than the total number of participants) for 
each group. 

n/a=data are not available (Group 4 is entering Grade 3 in 1996-97) 

t The year in which the students were at this grade level was when the labor negotiations and 
teachers’ strike occurred. 

tt Fifteen of the 103 discontinued students in Group 4 did not have midyear academic marks in 
Grade 2. Seven left the district after Grade 1, four left their schools or the district during Grade 2, 
three changed schools and their teachers did not issue marks for these students, and one moved 
into the district after midyear marks already had been issued. 



Also shown in Table 10, the percentage of discontinued students in Group 1 who earned at least 
“moderately experienced” in third-grade reading remained the same when compared with their 
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performance in second grade. The corresponding percentages for Group 2 represented an 
increase of 13 percentage points from second to third grade. For Group 3, by contrast, there was 
a decrease of seven percentage points from second to third grade. (Group 4 is entering third 
grade in 1996-97.) 

The attrition of eight students in Group 3 between second and third grade (six students who 
earned at least “moderately experienced” and two who earned below “moderately experienced” 
in second grade; see Table 10) did not solely account for the seven percentage point decline in 
reading described above. In addition to the attrition, five students in Group 3 who had earned at 
least “moderately experienced” in second grade declined below “moderately experienced” in 
third grade, while three others who had earned below “moderately experienced” in second grade 
improved to at least “moderately experienced” in third grade. Thus, individual students’ changes 
in performance also contributed to the decline. 

Approximately 80 percent of the discontinued students in Groups 1 and 2 earned at least 
“moderately experienced” in fourth-grade reading. More than 90 percent of those in Group 1 
earned at least “moderately experienced” in fifth-grade reading, as well. Group 3 is entering 
fourth grade in 1996-97. 

In summary, the results in Table 10 show that: 

• approximately six out of ten (113 of 188) discontinued students earned at least 
“moderately experienced” in second-grade reading (and eight out of ten, excluding 
Group 4); 

• approximately eight out of ten (68 of 86) discontinued students earned at least 
“moderately experienced” in third-grade reading; and 

• approximately eight out of ten (38 of 46) discontinued students earned at least 
“moderately experienced” in fourth-grade reading. 

Three additional data analyses were performed to find an explanation for why the discontinued 
students in Group 4 did not perform as well in second-grade reading as their counterparts in 
Groups 1, 2, and 3 (shown in Table 10). 

The first analysis examined the potential role of the students’ English fluency. Since 1994-95 
was the first year that the district had bilingual RR teachers, the possibility existed that a large 
proportion of English Language Learners (i.e., students who had not been reclassified as fluent in 
English) participated and that they were responsible for Group 4 discontinued students’ 
relatively lower performance in second-grade reading. If so, then a breakout of Group 4 
discontinued students by language level should reveal a large number of nonreclassified 
English Language Learners belonging to that group. 
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Table 11 shows a breakout of Group 4 discontinued students by language level. Only ten of 
these 88 students (11.4 percent) were English Language Learners, and three of them earned at 
least “moderately experienced” in second-grade reading. As previously shown in Table 10, the 
overall percentage who earned at least “moderately experienced” was 42.0 percent; it would have 
increased to only 43.6 percent if the ten English Language Learners had been excluded from 
Group 4. The similarity between these percentages (42.0 and 43.6) implies that Group 4 
discontinued students’ low performance in second-grade reading could not be attributable to the 
first-time inclusion of students who required native language tutoring during 1994-95. 



TABLE 1 1 

GROUP 4 DISCONTINUED STUDENTS’ MIDYEAR ACADEMIC PERFORMANCE 

IN SECOND-GRADE READING 



LANGUAGE LEVEL 


N 


ACHIEVED AT LEAST 
“MODERATELY EXPERIENCED” 






n 


% 


English proficient 


78 


34 


43.6 


|EngUsh Language 


w 


3 


33.3 


TOTAL 


88 


37 


42.0 



N=total number of discontinued students in Group 4 
n=number of students 

+ Values exclude students who had been reclassified as fluent in English. 



The second analysis addressed the possibility that the second-grade reading achievement of 
Group 4 reflected a districtwide decline in reading during 1994-95. Using preliminary data for 
the 1995-96 16 Expectations report, we found that the percentage of students districtwide who 
earned at least “moderately experienced” in third-grade reading was virtually identical 
(72.8 percent) to the corresponding value in 1994-95 (72.7 percent). Moreover, at four of the 
five sites that in 1995-96 had the largest numbers of second-grade students who had participated 
in RR during 1994-95, the percentage who earned at least “moderately experienced” in 
third-grade reading in 1995-96 was 2-9 percent higher than in 1994-95. Even though these 
findings pertain to third-grade students, they are indicative that Group 4’s second-grade 
performance was not part of a general decline in reading achievement districtwide during 
1994-95. 

The third analysis examined the possibility that students who had been tutored at a new RR site 
did not perform as well in second-grade reading as counterparts who had been tutored at a site 
that implemented RR prior to 1994-95. Of the 103 discontinued students in Group 4, all but four 
attended a new RR site. An analysis of how many students were tutored by a new RR teacher 
was beyond the scope of the present study. 

In summary, the three additional analyses provide evidence that the discrepancy between the 
second-grade reading achievement of Group 4 and that of Groups 1, 2, and 3 most likely is not 
attributable to the students’ lack of fluency in English, or to a districtwide decline in reading 
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achievement in 1994-95, or to the sites at which the students participated in the program. 
Group 4’s performance will be discussed in further detail later in this report. 

To provide a more detailed picture of the data in Table 10, Figures 2-5 show frequency 
distributions of the discontinued students’ midyear reading marks in second grade and above. 
These figures graphically represent the numbers of discontinued students who earned each of 
the following categories of academic marks in reading: “emergent reader,” 

“inexperienced reader,” “less experienced reader,” “moderately experienced reader,” 
“experienced reader,” and “exceptionally experienced reader.” Figure 2 presents these data for 
Group 1 (1991-92 participants), Figure 3 for Group 2 (1992-93 participants), Figure 4 for 
Group 3 (1993-94 participants), and Figure 5 for Group 4 (1994-95 participants). 

As shown in Figure 2, none of the discontinued students in Group 1 (1991-92 participants) 
earned “emergent reader” and only three earned “inexperienced reader” (one student in Grade 2 
and two in Grade 4). The number who earned “less experienced” increased by one in Grade 3 
and then decreased by two in Grade 4. The number who earned “moderately experienced” was 
the same in Grades 2 and 3, then decreased by two in Grade 4 and increased by one in Grade 5. 
Five discontinued students in Group 1 earned “experienced” in Grades 2, 3, and 4. 
Two discontinued students in Group 1 earned “exceptionally experienced” in Grade 5. 




ACADEMIC MARK IN READING 



Figure 2. Numbers of Discontinued Students in Group 1 Earning Each Category of Academic Marks in Reading in 
Grades 2 (1992-93), 3 (1993-94), 4 (1994-95), and 5 (1995-96). N=total number of discontinued students. See the 
above text for the complete names of the categories. 
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As shown in Figure 3, only one of the discontinued students in Group 2 earned “emergent 
reader” (Grade 4) and only one earned “inexperienced reader” (Grade 3). The number who 
earned “less experienced” decreased by seven (almost 80 percent) in Grade 3 and then increased 
by two in Grade 4. The number who earned “moderately experienced” increased by one in 
Grade 3 then decreased by nine (almost 40 percent) in Grade 4. Eleven discontinued students in 
Group 2 earned “experienced” in Grade 2, decreasing by three (almost 30 percent) in Grade 3, 
and increasing by four in Grade 4. Two discontinued students in Group 2 earned “exceptionally 
experienced” in Grade 3, but this number decreased to one in Grade 4. 




ACADEMIC MARK IN READING 



Figure 3. Numbers of Discontinued Students in Group 2 Earning Each Category of Academic Marks in Reading in 
Grades 2 (1993-94), 3 (1994-95), and 4 (1995-96). N=total number of discontinued students. See narrative on page 
15 for the complete names of the categories. 
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As shown in Figure 4, none of the discontinued students in Group 3 earned “emergent reader” in 
Grade 2 or 3, and only one earned “inexperienced reader” (Grade 2). One student was “not 
evaluated at this time” (not included in the graph). The number who earned “less experienced” 
increased by two (20 percent) in Grade 3, while the number who earned “moderately 
experienced” decreased by three (almost 17 percent) in Grade 3. Twelve discontinued students 
in Group 3 earned “experienced” in Grade 2, decreasing by seven (almost 60 percent) in Grade 3. 
Two discontinued students in Group 3 earned “exceptionally experienced” in Grade 3. 




ACADEMIC MARK IN READING 



Figure 4. Numbers of Discontinued Students in Group 3 Earning Each Category of Academic Marks in Reading in 
Grades 2 (1994-95) and 3 (1995-96). N=total number of discontinued students. See page 15 for the complete names 
of the categories. 
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As shown in Figure 5, four of the discontinued students in Group 4 earned “emergent reader” 
and ten earned “inexperienced reader” in Grade 2. The number who earned “less experienced” 
was almost 40 percent greater that the corresponding number who earned “moderately 
experienced” in Grade 2. Only nine (about 10 percent) of the discontinued students in Group 4 
earned “experienced” in Grade 2, and one earned “exceptionally experienced.” 




ACADEMIC MARK IN READING 



Figure 5. Numbers of Discontinued Students in Group 4 Earning Each Category of Academic Marks in Reading in 
Grade 2 (1995-96). N=total number of discontinued students. See page 15 for the complete names of the 
categories. 
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• Writing: The midyear marks of discontinued students in second*, third*, fourth-, and 
fifth-grade writing are presented in Table 12. This table shows the numbers and percentages of 
discontinued students who earned at least “satisfactory” in writing. 

Some students did not have marks because 1) they left the district before completing the grading 
period, or 2) they had multiple enrollments and were not present at a school long enough to earn 
marks. Only the ones who had midyear marks were included in the following analyses. 

As shown in Table 12, more than 75 percent of the discontinued students in each group earned 
at least “satisfactory” in second-grade writing. The percentage who earned at least “satisfactory” 
in second-grade writing decreased by about seven percentage points in 1994-95 (Group 3), 
relative to 1993-94 (Group 2), and decreased by an additional 13 percentage points in 1995-96 
(Group 4). 



TABLE 12 

DISCONTINUED STUDENTS’ MIDYEAR WRITING ACHIEVEMENT 
IN SECOND GRADE AND ABOVE 



GROUP 


GRADE 


N 


ACHIEVED AT LEAST 
“SATISFACTORY” 








n 






2 


14 


13 


||f^ : ^92.9®^i^ 


1 (RR in 1991-92; 


3 


16 


13 




50 participants; 19 discontinued) 


4 


14 


13 






-H- 


13 


12 




2 (RR in 1992-93; 

80 participants; 59 discontinued) 


2 


42 


41 


r^r?^ rf -97-6^-' 


3 


36 


31 




-H- 


32 


28 




3 (RR in 1993-94; 


2 


41 


37 


^/-vA.90.2 v r-f?;. 


79 participants; 49 discontinued) 




34 


25 




4 (RR in 1994-95; 


2i 


88±± 


68 


'S' - S * r-77 . 3 : .S - u ‘ ■ * Vv-V -I 


193 participants; 103 discontinued) 


3 


n/a 


n/a 





N=number of discontinued students in each group for whom academic marks were available (the 
different values of N across grade levels within groups were due to students leaving the district 
temporarily or permanently). Values include students attending a site (i.e., Damall) with a 
grade-reporting waiver and its own three-category rubric for marks in writing. 
n=number of discontinued students 

* These percentages are relative to the value of N (rather than the total number of participants) for 
each group. 

n/a=data are not available (Group 4 is entering Grade 3 in 1996-97) 

t The year in which the students were at this grade level was when the labor negotiations and 
teachers’ strike occurred. 

tt Fifteen of the 103 discontinued students in Group 4 did not have midyear academic marks in 
Grade 2. Seven left the district after Grade 1, four left their schools or the district during Grade 2, 
three changed schools and their teachers did not issue marks for these students, and one moved 
into the district after midyear marks already had been issued. 
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Also shown in Table 12, more than 80 percent of the discontinued students in Groups 1 and 2 
and less than 75 percent of those in Group 3 earned at least “satisfactory” in third-grade writing; 
these percentages for third-grade writing were lower than the corresponding ones for 
second-grade writing. The percentage of discontinued students earning at least “satisfactory” in 
third-grade writing increased by about five percentage points from 1993-94 (Group 1) to 
1994-95 (Group 2), but then decreased by about 12.5 percentage points in 1995-96 (Group 3). 
Group 4 is entering third grade in 1996-97. 

More than 85 percent of the discontinued students in Groups 1 and 2 earned at least 
“satisfactory” in fourth-grade writing. Over 90 percent of those in Group 1 earned at least 
“satisfactory” in fifth-grade writing, as well. Group 3 is entering fourth grade in 1996-97. 

In summary, the results in Table 12 show that: 

• approximately nine out of ten (159 of 185) discontinued students earned at least 
“satisfactory” in second-grade writing; 

• approximately eight out of ten (69 of 86) discontinued students earned at least 
“satisfactory” in third-grade writing; and 

• approximately nine out of ten (41 of 46) discontinued students earned at least 
“satisfactory” in fourth-grade writing. 
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• Spelling: The midyear marks of discontinued students in second-, third-, fourth-, and 
fifth-grade spelling are presented in Table 13. This table shows the numbers and percentages of 
discontinued students who earned at least “satisfactory” in spelling, plus the number who earned 
less than “satisfactory.” 

Some students did not have marks because 1) they left the district before completing the grading 
period, or 2) they had multiple enrollments and were not present at a school long enough to earn 
marks. Only the ones who had midyear marks were included in the following analyses. 

As shown in Table 13, more than 70 percent of the discontinued students in Groups 1-3 and 
only 64 percent in Group 4 earned at least “satisfactory” in second-grade spelling. The 
percentage who earned at least “satisfactory” in second-grade spelling increased by almost 
eight percentage points in 1993-94 (Group 2), relative to 1992-93 (Group 1), and decreased by 
about 20 percentage points in 1995-96 (Group 4), relative to 1994-95 (Group 3). 



TABLE 13 

DISCONTINUED STUDENTS’ MIDYEAR SPELLING ACHIEVEMENT 
IN SECOND GRADE AND ABOVE 



GROUP 


GRADE 


N 


ACHIEVED AT LEAST 
“SATISFACTORY” 


n 




1 (RR in 1991-92; 

50 participants; 19 discontinued) 


2 


14 


11 




3 


16 


13 


8 E SI 


4 


14 


10 




5± 


14 


13 




2 (RR in 1992-93; 

80 participants; 59 discontinued) 


2 


42 


34 




3 


36 


32 




4* 


31 


27 




3 (RR in 1993-94; 

79 participants; 49 discontinued) 


2 


39 


33 




n 


34 


27 




4 (RR in 1994-95; 

193 participants; 103 discontinued) 


2t 


86t* 


55 




3 


n/a 


n/a 





N=number of discontinued students in each group for whom academic marks were available (the 
different values of N across grade levels within groups were due to students leaving the district 
temporarily or permanently). Values do not include students attending a site (i.e., Damall) that did not 
issue marks for spelling. 
n=number of discontinued students 

* These percentages are relative to the value of N (rather than the total number of participants) for each 
group. 

n/a=data are not available (Group 4 is entering Grade 3 in 1996-97) 

$ The year in which the students were at this grade level was when the labor negotiations and teachers’ 
strike occurred. 

tt Fifteen of the 103 discontinued students in Group 4 did not have midyear academic marks in Grade 2. 
Seven left the district after Grade 1, four left their schools or the district during Grade 2, three changed 
schools and their teachers did not issue marks for these students, and one moved into the district after 
midyear marks already had been issued. 
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Also shown in Table 13, about 80 percent or more of the discontinued students in Groups 1-3 
earned at least “satisfactory” in third-grade spelling. The percentage of discontinued students 
earning at least “satisfactory” in third-grade spelling increased by about eight percentage points 
from 1993-94 (Group 1) to 1994-95 (Group 2), but then decreased by 9.5 percentage points in 
1995-96 (Group 3). Group 4 is entering third grade in 1996-97. 

More than 70 percent of the discontinued students in Groups 1 and 2 earned at least 
“satisfactory” in fourth-grade spelling. The percentage of discontinued students earning at least 
“satisfactory” in fourth-grade spelling increased by almost 16 percentage points from 1993-94 
(Group 1) to 1994-95 (Group 2). Over 90 percent of those in Group 1 earned at least 
“satisfactory” in fifth-grade spelling, as well. Group 3 is entering fourth grade in 1996-97. 

In summary, the results in Table 13 show that: 

• approximately seven out of ten (133 of 181) discontinued students earned at least 
“satisfactory” in second-grade spelling; 

• approximately eight out of ten (72 of 86) discontinued students earned at least 
“satisfactory” in third-grade spelling; and 

• approximately eight out of ten (37 of 45) discontinued students earned at least 
“satisfactory” in fourth-grade spelling. 

Academic Performance — Not Discontinued Students : The following sections present 
longitudinal data for not discontinued students (i.e., ones who did not attain their first-grade 
classmates’ average reading level). 
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• Reading: The midyear marks of not discontinued students in second-, third-, fourth-, and 
fifth-grade reading are presented in Table 14. This table shows the numbers and percentages of 
not discontinued students who earned at least “moderately experienced” in reading, plus the 
number who earned less than “moderately experienced.” 

Some students did not have midyear marks because 1) they left the district before completing the 
grading period, or 2) they had multiple (brief) enrollments and were not at a school sufficiently 
long to be issued marks by their teachers. Only the not discontinued students who had 
midyear marks were included in the following analyses. 

As shown in Table 14, more than 50 percent of the not discontinued students in Groups 1 and 2 
earned at least “moderately experienced” in second-grade reading. By contrast, only 15.8 and 
11.6 percent of the not discontinued students in Groups 3 and 4, respectively, earned at least 
“moderately experienced.” The reason(s) for this sharp decline is(are) not clear from the data. 



TABLE 14 

NOT DISCONTINUED STUDENTS’ MIDYEAR READING ACHIEVEMENT 
IN SECOND GRADE AND ABOVE 



GROUP 


GRADE 


N 


ACHIEVED AT LEAST 
“MODERATELY EXPERIENCED” 


n 




1 (RR in 1991-92; 

50 participants; 31 not discontinued) 


2 


24 


13 




3 


22 


9 




4 


24 


14 




5* 


20 


11 




2 (RR in 1992-93; 

80 participants; 21 not discontinued) 


2 


13 


8 


1?5 


3 


11 


7 




4* 


11 


6 




3 (RR in 1993-94; 

79 participants; 30 not discontinued) 


2 


19 


3 




3* 


19 


2 


SfevSSfl 0;5&% ££ rvxi 


4 (RR in 1994-95; 

193 participants; 90 not discontinued) 


2J 


69 


8 




3 


n/a 


n/a 





N=total number of not discontinued students in each group for whom academic marks were available 
(the different values of N across grade levels within groups were due to students leaving the district 
temporarily or permanently). Values include students attending a site (i.e., Damall) with a 
grade-reporting waiver and its own five-category rubric for marks in reading. 
n=number of not discontinued students 

* These percentages are relative to the value of N (rather than the total number of participants) for each 
group. 

$ The year in which the students were at this grade level was when the labor negotiations and teachers’ 
strike occurred. 

n/a=data are not available (Group 4 is entering Grade 3 in 1996-97) 



Also shown in Table 14, more than 40 percent of the not discontinued students in Groups 1 and 
2 earned at least “moderately experienced” in third-grade reading; fewer of these students in 
Group 1 earned at least “moderately experienced” in third grade than in second grade. The 



1995- 96 Report on the Reading Recovery Program 
Page 24 

percentage of such students earning at least “moderately experienced” in third-grade reading 
increased from 1993-94 (Group 1) to 1994-95 (Group 2). However, only 10.5 percent of the 
not discontinued students in Group 3 earned at least “moderately experienced” in third-grade 
reading. Again, the reason(s) for this sharp decline is(are) not clear. Group 4 will be in 
third grade in 1996-97. 

Over 50 percent of the not discontinued students in Groups 1 and 2 earned at least “moderately 
experienced” in fourth-grade reading. More than 50 percent of those in Group 1 earned at least 
“moderately experienced” in fifth-grade reading, as well. Group 3 is entering fourth grade in 

1996- 97. 

In summary, the results in Table 14 show that: 

• approximately three out of ten (32 of 125) not discontinued students earned at least 
“moderately experienced” in second-grade reading (and four out of ten, excluding 
Group 4); 

• approximately four out of ten (18 of 52) not discontinued students earned at least 
“moderately experienced” in third-grade reading; and 

• approximately six out of ten (20 of 35) not discontinued students earned at least 
“moderately experienced” in fourth-grade reading. 
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To provide a more detailed picture of the data in Table 14, Figures 6-9 show frequency 
distributions of the not discontinued students’ midyear reading marks in second grade and 
above. These figures graphically represent the numbers of not discontinued students who 
earned each of the following categories of academic grades in reading: “emergent reader,” 
“inexperienced reader,” “less experienced reader,” “moderately experienced reader,” 
“experienced reader,” and “exceptionally experienced reader.” Figure 6 presents these data for 
Group 1 (1991-92 participants). Figure 7 for Group 2 (1992-93 participants). Figure 8 for 
Group 3 (1993-94 participants), and Figure 9 for Group 4 (1994-95 participants). 



As shown in Figure 6, two of the not discontinued students in Group 1 (1991-92 participants) 
earned “emergent reader” in Grade 2 but none earned that mark in Grades 3, 4, or 5. Three of 
the not discontinued students in Group 1 earned “inexperienced reader” in Grade 2, and an 
additional student earned that mark in Grade 3. The number who earned “less experienced 
reader” in Grade 2 increased by three (50 percent) in Grade 3, then decreased back to six in 
Grade 4, and decreased again in Grade 5. Ten of the not discontinued students in Group 1 
earned “moderately experienced” in Grade 2, but that number decreased by four (40 percent) in 
Grade 3, increased by five (about 83 percent) in Grade 4, and decreased again (about 27 percent) 
in Grade 5. The number of not discontinued students in Group 1 who earned “experienced 
reader” was the same (3) in Grades 2 through 5. None of the not discontinued students in 
Group 1 earned “exceptionally experienced.” 
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ACADEMIC GRADE IN READING 



Figure 6. Numbers of Not Discontinued Students in Group 1 Earning Each Category of Academic Marks in 
Reading in Grades 2 (1992-93), 3 (1993-94), 4 (1994-95), and 5 (1995-96). N=total number of not discontinued 
students. See text above for the complete names of the categories. 
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As shown in Figure 7, one of the not discontinued students in Group 2 (1992-93 participants) 
earned “emergent reader” in Grade 2, one earned “inexperienced reader” in Grades 2 and 4, but 
none earned those marks in Grade 3. Three of the not discontinued students in Group 2 earned 
“less experienced reader” in Grade 2, and an additional student earned that mark in Grades 3 and 
4. Eight of the not discontinued students in Group 2 earned “moderately experienced” in 
Grade 2, but that number decreased by four (50 percent) in Grade 3 and increased by one in 
Grade 5. None of the not discontinued students in Group 2 earned “experienced” or 
“exceptionally experienced” in Grade 2, but two earned “experienced” in Grade 3 and one 
earned “experienced” in Grade 4. One student earned “exceptionally experienced” in Grade 3, 
while none earned that mark in Grades 2 and 4. 




ACADEMIC GRADE IN READING 



Figure 7. Numbers of Not Discontinued Students in Group 2 Earning Each Category of Academic Marks in 
Reading in Grades 2 (1993-94), 3 (1994-95), and 4 (1995-96). N=total number of not discontinued students. See 
text on page 25 for the complete names of the categories. 
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As shown in Figure 8, two of the not discontinued students in Group 3 (1993-94 participants) 
earned “emergent reader” in Grades 2 and 3, while five earned “inexperienced reader” in 
Grades 2 and 3. Nine of the not discontinued students in Group 3 earned “less experienced” in 
Grade 2; that number increased by one in Grade 3. Three earned “moderately experienced” in 
Grade 2, and that number decreased by one in Grade 3. None of the not discontinued students 
in Group 3 earned “experienced” or “exceptionally experienced.” 




ACADEMIC GRADE IN READING 



Figure 8. Numbers of Not Discontinued Students in Group 3 Earning Each Category of Academic Marks in 
Reading in Grades 2 (1994-95) and 3 (1995-96). N=total number of not discontinued students. See text on page 25 
for the complete names of the categories. 
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As shown in Figure 9, 14 (20.3 percent) of the not discontinued students in Group 4 (1994-95 
participants) earned “emergent reader” and 24 (34.8 percent) earned “inexperienced reader” in 
Grade 2. The number who earned “less experienced” was three times greater that the 
corresponding number who earned “moderately experienced” in Grade 2. None of the not 
discontinued students in Group 4 earned “experienced” in Grade 2, and only one earned 
“exceptionally experienced.” 




ACADEMIC GRADE IN READING 



Figure 9. Numbers of Not Discontinued Students in Group 4 Earning Each Category of Academic Marks in 
Reading in Grade 2 (1995-96). Two students were not evaluated (not shown in the graph). N=total number of not 
discontinued students. See text on page 25 for the complete names of the categories. 
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• Writing: The midyear marks of not discontinued students in second-, third-, fourth-, and 
fifth-grade writing are presented in Table 15. This table shows the numbers and percentages of 
not discontinued students who earned at least “moderately experienced” in writing, plus the 
number who earned less than “moderately experienced.” 

Some students did not have midyear marks because 1) they left the district before completing the 
grading period, or 2) they had multiple (brief) enrollments and were not at a school sufficiently 
long to be issued marks by their teachers. Only the not discontinued students who had midyear 
marks were included in the following analyses. 

As shown in Table 15, more than 70 percent of the not discontinued students in Group 1 earned 
at least “satisfactory” in second-grade writing. By contrast, only 53.8, 45 and 48.5 percent of the 
not discontinued students in Groups 2, 3 and 4 (respectively) earned at least “satisfactory.” The 
reason(s) for this decline is(are) not clear from the data. 



TABLE 15 

NOT DISCONTINUED STUDENTS’ MIDYEAR WRITING ACHIEVEMENT 
IN SECOND GRADE AND ABOVE 



GROUP 


GRADE 


N 


ACHIEVED AT LEAST 
“SATISFACTORY” 


n 




1 (RR in 1991-92; 

50 participants; 31 not discontinued) 


2 


22 


16 




3 


22 


12 




4 


24 


18 




5t 


19 


11 


fe:5 7 . 9- :l N-i 


2 (RR in 1992-93; 

80 participants; 21 not discontinued) 


2 


13 


7 




3 


11 


6 




4$ 


11 


8 




3 (RR in 1993-94; 

79 participants; 30 not discontinued) 


2 


20 


9 


i^v ??'745.0 


3* 


19 


11 




4 (RR in 1994-95; 1 

193 participants; 90 not discontinued) 


2* _ 


68 


33 


^'••/X>M8.5v ::'Ja r4 


3 


n/a 


n/a 





N=total number of not discontinued students in each group for whom academic marks were available 
(the different values of N across grade levels within groups were due to students leaving the district 
temporarily or permanently). Values include students attending a site (i.e., Damall) with a 
grade-reporting waiver and its own three-category rubric for marks in writing. 
n=number of not discontinued students 

* These percentages are relative to the value of N (rather than the total number of participants) for each 
group. 

t The year in which the students were at this grade level was when the labor negotiations and teachers’ 
strike occurred. 

n/a=data are not available (Group 4 is entering Grade 3 in 1996-97) 



Also in Table 15, approximately 55 percent of the not discontinued students in Groups 1 and 2 
earned at least “satisfactory” in third-grade writing; fewer of these students in Group 1 earned at 
least “satisfactory” in third grade than in second, grade, and about the same percentage of these 
students in Group 2 earned at least “satisfactory” in second and third grades. The percentage of 
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not discontinued students earning at least “satisfactory” in third-grade writing was identical in 
1993-94 (Group 1) and 1994-95 (Group 2). A similar percentage (approximately 58 percent) of 
the not discontinued students in Group 3 earned at least “satisfactory” in third-grade writing. 
Group 4 is entering third grade in 1996-97. 

Over 70 percent of the not discontinued students in Groups 1 and 2 earned at least “satisfactory” 
in fourth-grade writing (Table 15). More than 50 percent of those in Group 1 earned at least 
“satisfactory” in fifth-grade writing, as well. Group 3 is entering fourth grade in 1996-97. 

In summary, the results in Table 15 show that: 

• approximately five out of ten (65 of 123) not discontinued students earned at least 
“satisfactory” in second-grade writing; 

• approximately six out of ten (29 of 52) not discontinued students earned at least 
“satisfactory” in third-grade writing; and 

• approximately seven out of ten (26 of 35) not discontinued students earned at least 
“satisfactory” in fourth-grade writing. 
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• Spelling: The midyear marks of not discontinued students in second-, third-, fourth-, and 
fifth-grade spelling are presented in Table 16. This table shows the numbers and percentages of 
not discontinued students who earned at least “satisfactory” in spelling, plus the number who 
earned less than “satisfactory.” 

Some students did not have midyear marks because 1) they left the district before completing the 
grading period, or 2) they had multiple (brief) enrollments and were not at a school sufficiently 
long to be issued marks by their teachers. Only the not discontinued students who had midyear 
marks were included in the following analyses. 

As shown in Table 16, more than 55 percent of the not discontinued students in Groups 1 and 2 
earned at least “satisfactory” in second-grade spelling. By contrast, only about 33 percent and 
46 percent of the not discontinued students in Groups 3 and 4 earned at least “satisfactory.” 
The reason(s) for this decline is(are) not clear from the data. 



TABLE 16 

NOT DISCONTINUED STUDENTS’ MIDYEAR SPELLING ACHIEVEMENT 
IN SECOND GRADE AND ABOVE 



GROUP 


GRADE 


N 


ACHIEVED AT LEAST 
“SATISFACTORY” 








n 






2 


24 


14 




1 (RR in 1991-92; 


3 


22 


12 




50 participants; 31 not discontinued) 


4 


24 


10 






5* 


20 


20 


P^P§5®1 00. 


2 (RR in 1992-93; 

80 participants; 21 not discontinued) 


2 


13 


8 


&$?%£} ' .61 ;'5 


3 


11 


6 




4$ 


11 


7 




3 (RR in 1993-94; 


2 


18 


6 




79 participants; 30 not discontinued) 


3t 


18 


10 




4 (RR in 1994-95; 


2* 


67 


31 




193 participants; 90 not discontinued) 


3 


n/a 


n/a 





N=total number of not discontinued students in each group for whom academic marks were available 
(the different values of N across grade levels within groups were due to students leaving the district 
temporarily or permanently). Values do not include students attending a site (i.e., Damall) that did not 
issue marks for spelling. 
n=number of not discontinued students 

* These percentages are relative to the value of N (rather than the total number of participants) for each 
group. 

t The year in which the students were at this grade level was when the labor negotiations and teachers’ 
strike occurred. 

n/a=data are not available (Group 4 is entering Grade 3 in 1996-97) 



Also in Table 16, approximately 55 percent of the not discontinued students in Groups 1, 2, and 
3 earned at least “satisfactory” in third-grade spelling. Group 4 will be in third grade in 1996-97. 
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Over 40 percent of the not discontinued students in Groups 1 and 2 earned at least “satisfactory” 
in fourth-grade spelling (Table 16). All students in Group 1 earned at least “satisfactory” in 
fifth-grade spelling, as well. Group 3 is entering fourth grade in 1996-97. 

In summary, the results in Table 16 show that: 

• approximately five out of ten (59 of 122) not discontinued students earned at least 
“satisfactory” in second-grade spelling; 

• approximately six out of ten (28 of 51) not discontinued students earned at least 
“satisfactory” in third-grade spelling; and 

• approximately five out of ten (17 of 35) not discontinued students earned at least 
“satisfactory” in fourth-grade spelling. 

Participation in Additional Compensatory Programs : To determine whether participation in the 
RR program reduced students’ need for additional compensatory services (and thereby provided 
a financial benefit to the district), an analysis was performed on the number of participants 
assigned to Special Education 2 in first, second, or third grade. Students who left the district were 
not included in the following analyses. 



2 Chapter I/Title I participation was not analyzed in the present report. The reason was that most 
RR students probably received Chapter I/Title I services in second and third grades simply 
because the overwhelming majority of RR sites were schoolwide Chapter I/Title I. It was 
beyond the scope of this study to find out the percentage of RR participants who actually 
required Chapter I/Title I services. 



1995- 96 Report on the Reading Recovery Program 
Page 33 

• Discontinued Students: The numbers and percentages of discontinued students receiving 
Special Education services in first, second, and/or third grade are presented in Table 17. Out of 
the 230 participants who were discontinued from the RR program between 1991-92 and 
1994—95, 11 (4.8 percent) received Special Education services in first grade. Twelve 
discontinued students also received such services in second grade. The number who received 
Special Education services in third grade was the same as in second grade, excluding Group 4. 
Third-grade data for Group 4 were unavailable (these students are entering third grade in 

1996- 97). 



TABLE 17 

DISCONTINUED STUDENTS RECEIVING SPECIAL EDUCATION SERVICES 



GROUP 


GRADE 


N 


SPECIAL EDUCATION 


n 


% 


1 (RR in 1991-92; 

50 participants; 19 discontinued) 


1 


19 


0 


0 


2 


16 


0 


0 


3 


16 


0 


0 


2 (RR in 1992-93; 

80 participants; 59 discontinued) 


1 


59 


1 


1.7 


2 


42 


1 


2.4 


3 


36 


1 


2.8 


3 (RR in 1993-94; 

79 participants; 49 discontinued) 


1 


49 


4 


8.2 


2 


42 


3 


7.1 


3 


34 


3 


8.8 


4 (RR in 1994-95; 

193 participants; 103 discontinued) 


1 


103 


6 


5.8 


2 


88 


8 


9.1 


3 


n/a 


n/a 


n/a 



N=number of discontinued students in each group for whom midyear marks in reading were available. 
The value for first grade represents the total number of discontinued students for each group. 
n=number of discontinued students who received Special Education services 
n/a=data are not available (Group 4 is entering Grade 3 in 1996-97) 
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• Not Discontinued Students: The numbers and percentages of not discontinued students 
receiving Special Education services in first, second, and/or third grade are presented in 
Table 18. Out of the 172 participants who were not discontinued from the RR program between 
1991-92 and 1994-95, 11 (6.4 percent) received Special Education services in first grade. 
Twenty-four not discontinued students also received such services in second grade. 
Four additional not discontinued students received Special Education services in third grade 
than in second grade, excluding Group 4. Third-grade data for Group 4 were unavailable (these 
students are entering third grade in 1996-97). 



TABLE 18 

NOT DISCONTINUED STUDENTS RECEIVING SPECIAL EDUCATION SERVICES 



GROUP 


GRADE 


N 


SPECIAL EDUCATION 


n 


% 


1 (RRin 1991-92; 

50 participants; 19 discontinued) 


1 


31 


2 


6.4 


2 


24 


4 


16.7 


3 


22 


5 


22.7 


2 (RR in 1992-93; 

80 participants; 59 discontinued) 


1 


21 


1 


4.8 


2 


13 


4 


30.8 


3 


11 


5 


45.4 


3 (RR in 1993-94; 

79 participants; 49 discontinued) 


1 


30 


4 


13.3 


2 


19 


5 


26.3 


3 


19 


7 


36.8 


4 (RR in 1994-95; 

193 participants; 103 discontinued) 


1 


90 


4 


4.4 


2 


69 


ii 


15.9 


3 


n/a 


n/a 


n/a 



N=number of not discontinued students in each group for whom midyear marks in reading were 
available. The value for first grade represents the total number of not discontinued students for each 
group. 

n=number of not discontinued students who received Special Education services 
n/a=data are not available (Group 4 is entering Grade 3 in 1996-97) 
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Literature Review Update: The literature review in Report on the Reading Recovery Program in 
San Diego Unified School District (Evaluation Unit Report No. 705) summarized the findings of 
several studies that provided evidence for sustained effects of the RR program. Specifically, the 
studies found that the majority of discontinued students continued to perform at or above their 
classmates’ average level in second and third grades. 

During the past year, additional findings on RR’s sustained effects have appeared in the research 
literature. One study in particular (Center et al., 1995) was designed to address weaknesses and 
flaws in previous evaluations of RR’s sustained effects (e.g.. Clay, 1985; Pinnell et al., 1994). 

Center et al. studied ten Australian schools where low-achieving students were assigned 
randomly to either RR or a control condition in which they had access to any reading support 
(other than RR) typically available at their school. These students, as well as age-peers at 
selected comparison schools that did not have RR, were tested at four separate time points 
using standardized and criterion-referenced tools (Neale Analysis of Reading Ability, 
Passage Reading Test, Waddington Diagnostic Spelling Test, Phonemic Awareness Test, 
Cloze Test of Syntactic Awareness, and Word Attack Skills Test) plus Clay’s 
Observational Survey. 

The major findings of Center et al.’s (1995) study were: 

• at the pretest time point, the RR, control, and comparison students did not differ from 
each other on any of the measures; 

• at the posttest time point (15 weeks after RR intervention began), RR participants 
outperformed control and comparison counterparts on all tests except the ones measuring 
syntactic or phonemic awareness; 

• four months after the posttest, although RR participants continued to outperform control 
and comparison counterparts on all tests except the ones measuring syntactic awareness 
or phonological recoding, the size of the difference between the RR students and controls 
had diminished considerably; 

• 12 months after the posttest, RR participants outperformed the control and comparison 
students only on Clay’s book level test (however, at this time point, attrition had reduced 
the control group by 23 students — almost 60 percent — and the remaining students 
probably were skilled readers since 15 of the 23 had entered RR); 

• of the original 23 RR participants, 35 percent met the program’s criteria for 
discontinuation, another 30 percent were in the average range of their classmates but did 
not meet all criteria, while the remainder were not discontinued; and 

• of the 16 remaining control students, 37 percent had improved in the absence of 
individualized tutoring. 

These findings, taken together, were indicative that 35 percent of the RR participants had 
benefitted from the program, 30 percent probably would have improved on their own, and 
35 percent remained low achievers. Although Center et al. emphasized that their findings 
applied only to the RR program as implemented in Australia, they nevertheless cited RR’s 
considerable expense and the need to enhance its efficacy and cost-effectiveness. 

Less favorable findings were reported for the Wake County (North Carolina) Public School 
System’s RR program (Baenen et al., 1995). No difference was found between the percentage of 
RR participants in 1990-91 and 1991-92 who attained grade-level reading in third grade and the 
corresponding values for comparison students. Moreover, the percentage of RR students who 
were retained or required additional compensatory services (Chapter I/Title I; Special Education) 
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was essentially identical to that for the comparison group. RR costs were almost $2,947 per 
student above the regular instructional program. The authors concluded that RR “does not 
appear to be cost effective at this time.” (page 25) 

For a comprehensive and critical review of published and unpublished reports on RR, see 
Shanahan and Barr (1995). 

Financial Data 

RR appears to be an expensive program, primarily because it involves one-on-one tutoring and 
specialized training for participating teachers. The district’s program has expanded rapidly, and 
consequently the costs also have grown. 

The exact cost per student of the RR program is difficult to specify, primarily because of 
controversy over how to account for teachers’ salaries and benefits. According to program 
supporters, it is inappropriate to include full-time salaries in cost determinations because the 
teachers tutor RR students only half-time (RR teachers work with literacy groups of about 
15 students during the afternoon). Conversely, nonsupporters prefer to include full-time salaries 
and benefits in determining RR’s cost because pairs of teachers could (but do not) team-teach a 
kindergarten class during the time when they are not tutoring. 

Another reason for the difficulty in specifying the exact cost per RR student was that the 
External Funding Budget Office changed its accounting procedures for this program in 1994-95. 
Because of the program’s expansion that year, it became necessary to implement a single 
funding code for all RR teachers regardless of their actual funding source. In the first three years 
of the program, each RR teacher was paid full-time but only half of their funding was a central 
office cost and the other half was paid out of district general funds given to sites (AA funds). 
This procedural change led to an understatement of the RR teachers’ salaries and benefits for 
1991-92 through 1993-94 in Report on the Reading Recovery Program in San Diego Unified 
School District (Evaluation Unit Report No. 705; page 30). Corrected values have been obtained 
for the present report. 
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The financial data in Table 19, provided by the Finance Division’s External Funding Budget 
Office, reflect Chapter I/Title I funds actually spent by the central office for the RR program; 
these data do not reflect any expenditures from the Chapter I/Title I site formula allocation. Two 
cost estimates (one including half-time salaries and benefits; the other including full-time values) 
are presented, and data for 1995-96 are included (even though student achievement data are not 
yet available) in order to provide a comparison with the data for 1994-95 (see Discussion, page 
38). “Other” costs include teacher leaders’ salaries and benefits, travel expenses, supplies, staff 
development, tuition for RR training costs, and the like (an additional cost for 1995-96 included 
in Table 19 was the training of 1 1 bilingual teachers in the Spanish version of the program). 



TABLE 19 

ANNUAL CENTRAL OFFICE COSTS OF THE READING RECOVERY PROGRAM* 



YEAR 


COST CATEGORIES 


HALF-TIME 


FULL-TIME 


1991-92 


Teachers’ salaries & benefits 


237,331 


474,661 


Other 


45,000 


45,000 


Total cost 


282.331 


519.661 


Cost per RR student (N=50) 


5,647 


N/A 


1992-93 


Teachers’ salaries & benefits 


324,751 


649,501 


Other 


45,000 


45,000 


Total cost 


369.751 


694.501 


Cost per RR student (N=80) 


4,622 


N/A 


1993-94 


Teachers’ salaries & benefits 


264,711 


529,422 


Other 


185,000 


185,000 


Total cost 


449.711 


714.422 


Cost per RR student (N=79) 


5,693 


N/A 


1994-95 


Teachers’ salaries & benefits 


1,021,945 


2,043,890 


Other 


310.800 


310.080 


Total cost 


1.332.745 


2.353.970 


Cost per RR student (N=193) 


6,905 


N/A 


1995-96 


Teachers’ salaries & benefits 


1,400,406 


2,800,811 


Other 


303.967 


303.967 


Total cost 


1.704.373 


3.104.778 


Cost per RR student (N=440) 


3,874 


N/A 


fe%^/totaisS» 
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* All costs are in dollars. These data reflect only expenditures (JK or AA funds) that are not part of the 
Chapter I/Title I site formula allocation. “Other” costs include teacher leaders’ salaries and benefits 
(beginning in 1993-94, there were two teacher leaders and a third was added in 1994-95), training 
expenses, staff development, supplies, etc. 

* Some teachers were paid from site funds. The salary of one teacher who was hired on substitute status 
was not included in this table. 

N=number of participating students 

N/A=not applicable (RR teachers work half-time with literacy groups of about 15 students) 
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Summary of Findings 

To determine the degree to which the district’s RR program had sustained effects on reading 
achievement, the present study evaluated participants’ English-language arts performance in 
second grade and above. The study specifically examined the numbers and percentages of 
discontinued and not discontinued students who were graded “moderately experienced reader” 
by their classroom teacher (superintendent’s Expectation 1 — “All students mastering reading 
skills and comprehension by grade three . . 

The major findings of this study were as follows: 

• Short-term treatment effect 

1. The percentage of discontinued participants in 1992-93 and 1993-94 was consistent with 
corresponding percentages in other districts. However, it declined in San Diego Unified 
from 92 percent in 1993-94 to 82 percent in 1994-95. 

2. From 1992-93 through 1994-95, almost 50 percent of all not discontinued students still 
were in the program at the end of the school year with fewer than the required 60 lessons. 
Approximately 18 percent of the not discontinued students had completed at least 
60 lessons but still did not attain their classmates’ average reading level on the program’s 
assessment instrument. 

• Sustained treatment effect 

1. More than 75 percent of all discontinued students in 1991-92 and 1992-93 were 
“moderately experienced readers” in Grades 2, 3, and 4. 

2. Only about two-thirds of the discontinued students in 1993-94 were “moderately 
experienced readers” in Grades 2 and 3. 

3. Less than half of the discontinued students in 1994-95 were “moderately experienced 
readers” in Grade 2. 

• Central office costs 

1. Over the past five years, the costs have amounted to $4,916 per RR student (based upon 
teachers’ half-time salaries and benefits; RR teachers spend only half of their time 
tutoring participants). Total costs paid by sites are not known. 

2. The central office’s costs in 1994-95 were $6,905 per RR student (based upon teachers’ 
half-time salaries and benefits) — over $1,000 higher than the cost in any of the previous 
three years. In 1995-96, the cost decreased to $3,874 per RR student (based upon 
teachers’ half-time salaries and benefits) even though there were additional costs that 
year for training 1 1 bilingual RR teachers in the Spanish version of the program. The 
changes in cost occurred during rapid expansion of the program. In 1994-95, it expanded 
from 79 participants to 193 and from 14 teachers to 38 (including two teachers whose 
positions were funded by their sites). In 1995-96, it expanded from 193 participants to 
440 and from 38 teachers to 55.5. 

Thus, the program’s expansion has been accompanied by three important outcomes: a decrease 
in the percentage of students who are discontinued; a decrease in the percentage who are 
moderately experienced readers in Grade 2 and above; and an increase in the cost per 
RR student. 
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Discussion : This section considers potential explanations for the evaluation’s major findings. 

• Why Group 4 Declined: 1994-95 participants (Group 4) earned lower midyear marks in 
second-grade reading, writing, and spelling than previous years’ participants, yet their RR costs 
were approximately doubled. The present study attempted to explain this, but instead provided 
evidence that Group 4’s lower performance could not be attributed to the participants’ 
language level (see Table 11, page 14), a districtwide decline in reading achievement in 1994-95 
(see page 14), nor the sites that RR students attended (see page 14). 

Some other variable(s) must have been responsible for the performance decline and cost increase 
in 1994-95. The following known factors might have played a role: program expansion (from 
79 participants and 14 teachers in 1993-94 to 193 participants and 38 teachers in 1994-95); 
adoption of a more stringent student selection criterion (see page 1); disruptions due to the labor 
contract negotiations and strike in 1995-96 (which might have affected Group 4’s reading 
achievement during second grade); and interruptions due to the single track year-round calendar 
at some RR schools. 

Evidence for the possible role of program expansion is presented in Table 20. Even though the 
program’s goal was for each teacher to tutor eight students per year, the program’s teacher case 
load declined between 1992-93 and 1994-95 due (in part) to the additional teachers-in-training. 
The number of teachers increased in 1994-95 to a greater degree than the increase in the number 
of students; consequently, each teacher was tutoring an average of 5.08 students (the lowest 
value since the program’s inception in the district). 



TABLE 20 

TEACHERS AND STUDENTS IN THE READING RECOVERY PROGRAM* 



YEAR 


TEACHERS 


STUDENTS 


CASELOAD 


COST PER STUDENT** 




N 


% change 


N 


% change 


RATIO* 


% change 


$ 


% change 


1991-92 


8 


— 


50 


— 


6.25 


— 


5,647 


— 


1992-93 


13 


62.5 


80 


60.0 


6.15 


-1.6 


4,622 


-18.2 


1993-94 


14 


7.7 


79 


-1.25 


5.64 


1 

00 


5,693 


23.2 


R99SS51 


mmm 




Sii?3# 


lgi443|$ 


WMs-o&WB- 




wmmm 




1995-96 


55.5 


46.0 


440 


128.0 


7.93 


56.1 


3,874 


-43.9 



* Percentage changes are relative to the previous year’s value (a negative value represents a decrease). 

N= total number 

* Ratio=number of students per teacher 

** Values represent central office expenditures that are based upon teachers’ half-time salaries and benefits (see 
page 37). 

** Table 20 includes all RR teachers (including ones who were paid from site funds). 



Program expansion most likely did play an important role in the high cost per RR student in 
1994-95. Because the number of students per teacher that year was below the program’s goal of 
eight, it mathematically resulted in an inflated cost per RR student. (An important point to keep 
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in mind, however, is that the values in Tables 19 and 20 reflect central office costs — AA and JK 
funds — only. 3 ) Conversely, the case load per teacher increased in 1995-96 and the cost per RR 
student correspondingly declined. 

A more likely explanation for the decline in the percentage of discontinued participants in 
1994-95 is that it might have been an anomaly of the program’s selection criterion. In 1994-95, 
access to the program was restricted to the most needy students in the lowest 20 percent of their 
first-grade classmates while in the previous three years participants were selected from a range 
within the lowest 30 percent. The percentage of discontinued participants in 1994-95 therefore 
might have been more realistic than that in the previous years. 

It was beyond the scope of this study to identify and compare the second-grade marks of 
RR students who were tutored by one of the 24 new teachers in 1994-95 and the corresponding 
marks of counterparts who were tutored by one of the experienced teachers. Thus, the degree to 
which program expansion has played a role in the decline in RR students’ reading achievement 
in Grade 2 and above remains to be evaluated. 

• Evidence for Sustained Effects: The present study was designed to address the evaluation 
question “Do discontinued students maintain their acquired literacy skills in the second, third, 
and fourth grades?” 

As shown in Table 10 (page 12), Groups 1 and 2 provided evidence that RR did have sustained 
effects as reflected by the high percentages who earned at least “moderately experienced” in 
reading. (The values actually were higher than those for third grade students 
districtwide — 74.6 percent in 1993-94 and 72.7 percent in 1994-95 according to 16 Expectations 
Performance Indicators: 1994-95 Summary ; Research, Reporting, and Grants Unit Report 
No. 016.) Group 3 conversely did not have as high a percentage in third grade, although that 
might have been attributable to attrition (as shown in Table 10, there were eight fewer Group 3 
students who earned at least “moderately experienced” in third-grade reading, and there were 
eight fewer for whom academic marks were available). 

Overall, the findings of this study suggest that the answer to the above evaluation question is 
“yes” for many RR students. 

Although the present findings about RR’s sustained effects confirmed previously published ones 
(e.g., Pinnell et al., 1994), they were inconsistent with the findings of Center et al. (1995). The 
inconsistency probably was attributable (at least in part) to statistical methodology. In the 
present study, the discontinued and not discontinued students were analyzed separately — the 
students were divided into two subpopulations on the basis of their successful performance on 
the Observational Survey after completing 60+ lessons. In the study by Center et al., however, 
all participants were combined into one group (RR students). This statistical method was 
questionable because it “mixed apples and oranges.” Consequently, Center et al.’s conclusions 
about the program’s inadequacies might have been more appropriately restricted to the 
not discontinued students rather than generalized to all RR participants. 



3 Another important point to note about the program’s costs is that the district pays for the 
teachers’ RR training costs (college credits) and their full-time salaries while they are in training. 
Also, the teachers gain seniority time while they are in training. These additional costs are 
incorporated into Tables 19 and 20. By contrast, teachers who choose to earn college credits 
toward a higher degree pay their own tuition costs and take courses on their own time. 
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It might be worthwhile to consider whether questions about sustained effects should be used to 
evaluate RR. One problem with such questions is that they actually are inconsistent with the 
program’s design and stated goals. RR is intended to be a short-term (12-14 weeks) intervention 
that focuses on identifying and correcting improper reading strategies used by low-achieving 
students so that they can catch up to their classmates, read independently, and progress at an 
average rate (Clay, 1985; Pinnell, 1989; Pinnell et al., 1994; Shanahan and Barr, 1995). 
Moreover, RR is known to not work for every student 4 (Pinnell, 1989). 

Another problem with evaluation questions about RR’s sustained effects is that they presume RR 
to be a permanent solution for early reading difficulties. This presumption recently has been 
challenged in the research literature, however. 

To use a medical analogy, early interventions are supposed to operate like a 
vaccination, preventing all future learning problems no matter what their source or 
severity. It appears, however, that early interventions, no matter how successful, are 
more similar to insulin therapy. That is, substantial treatment effects are apparent 
right away, but these gains can be maintained only through additional intervention and 
support. . . . That the effects of Reading Recovery and other early interventions are 
apparent for so long without [additional] support is a testimonial to their quality. 
(Shanahan and Barr, 1995; page 982) 

If RR’s original design and stated goals are taken into consideration when formulating evaluation 
questions, it might be more appropriate to investigate the extent to which discontinued students 
read independently and progress at their classmates’ average rate in second, third, and 
fourth grade as compared to control students who are identical to discontinued counterparts in 
all respects, except that they have not participated in the program. 



Instructional Program Implications 

The present study has analyzed the status of participants in the district’s RR program from 1991- 

92 through 1994-95 and their midyear marks in English-language arts content areas during 

Grade 2 and above. Although there is some variability, the findings generally suggest that: 

1. the percentage of discontinued students has varied from year to year, perhaps as a result of 
the program’s problematic implementation (see page 1) and rapid expansion in the district; 

2. discontinued students (except for those in 1994-95) have shown sustained academic benefit 
from RR (i.e., at least 65 percent of these students were moderately experienced readers or 
better in Grade 2 and above); 

3. discontinued students have derived more instructional benefit from the program than 
not discontinued participants (higher percentages of the former were moderately 
experienced readers in Grade 2 and above than the latter); and 



4 An additional consideration is that American participants typically enter the program with 
much greater reading difficulties than New Zealand counterparts (RR was designed specifically 
for use in the educational system/culture of that country). The former have been shown to read 
1.46 words while the latter read 7.70. American students read at book level 1.33, New Zealand 
students at level 5.00. The data for writing vocabulary are 3.72 versus 8.75, and for concepts of 
print 8.32 versus 12.84 (Shanahan and Barr, 1995). Perhaps it is remarkable that RR has any 
impact on American students. 
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4. the program’s cost-benefit has been impacted negatively by the sizable number of 
participants who do not complete the required 60 RR lessons and consequently are 
not discontinued. 

Although this study did not include a control group, all participants served as their own controls 
in that their “baseline” reading performance (as measured by the program’s Observational 
Survey instrument) in first grade put them in the lowest 20 percent of their classmates. It is not 
known how many would have achieved “moderately experienced” or above if they had not 
participated; published research findings suggest that up to 30 percent might have improved 
without intervention (Center et al., 1995). 



Facilities Implications 

This report does not have any definitive facilities implications for the district. However, it is 
possible that continued expansion of the RR program during class-size reduction might result in 
additional demands for space. 



Budget Implications 

A sizable number of participants have not derived maximal benefit from the program because 
they have not completed the required 60 lessons. Such participants comprise almost half of the 
not discontinued students (20.4 percent of all participants). Thus, the district is investing 
heavily in students who do not participate in the program long enough to fully gain the potential 
benefits. 

The present report provided evidence that the cost per student increased dramatically when the 
teacher case load declined. This evidence might have implications for plans to further expand 
the program. 



Public Support and Engagement Implications 

Site staffs, governance teams, and school improvement councils should be informed about the 
findings of this report. 

Some of the present findings (e.g., academic data for 1994-95 participants and financial data) 
might have an impact on public perceptions and reactions toward the program. 



Recommendations 

The following actions are recommended for district staff: 



1. RR schools should identify and provide maintenance (including RR literacy groups) for 
participants who are not earning at least “moderately experienced” in reading beyond 
first grade. 
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2. Investigate and implement strategies to increase the percentage of participating students 
who remain in the program long enough to be discontinued. 

3. Ensure that case loads approximate eight students per year. 

4. Eliminate the delay in receiving preliminary data regarding the percentage of 
discontinued students. 

5. Continue to consider other literacy programs for low-achieving students that might impact a 
larger number of students, be more cost effective, and serve students beyond first grade. 

6. Explore the possibility of attaining full implementation at a limited number of sites (as a 
pilot for the district) in order to assess the program’s short-term and sustained effects under 
such conditions so that future decisions about improving and/or expanding the program 
can be facilitated. 

7. Appropriately budget for continued monitoring of the RR program’s costs and benefits. 



Report prepared by Ruben Carriedo/Barry Fass-Holmes and Frank Ciriza 
RC:bf-h 
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